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A DEGREE THEORY FOR SECOND ORDER NONLINEAR ELLIPTIC 
OPERATORS WITH NONLINEAR OBLIQUE BOUNDARY CONDITIONS 

YANYAN LI, JIAKUN LIU, AND LUC NGUYEN 
Dedicated to Paul H. Rabinowitz on his 75th birthday with admiration 


Abstract. In this paper we introduce an integer-valued degree for second order fully nonlinear 
elliptic operators with nonlinear oblique boundary conditions. We also give some applications 
to the existence of solutions of certain nonlinear elliptic equations arising from a Yamabe 
problem with boundary and reflector problems. 


1. Introduction 


Degree theories are very useful in the study of partial differential equations, for example, in 
the study of existence and multiplicities of solutions, eigenvalue and bifurcation problems. See 
for example liliaEolEIlES]. 

In [T3] . the first named author introduced a degree theory for second order nonlinear elliptic 
operators with Dirichlet boundary conditions. It is natural to ask for a degree theory for 
other boundary operators. Problems with nonlinear oblique boundary conditions have been 
considered in the literature for some time, see e.g. [HlElEllIlllIlllIBlinillHlIiniEsllSl]. 

For example, in the study of boundary Yamabe problems laEKiii [T5] . one considers the 
boundary condition 


( 1 . 1 ) 


hi := u "-2 




n — 2 
2 



= c 


on dM, 


where dM is the boundary of a smooth Riemannian manifold {M,g) of dimension n > 3, u 
is the outer unit normal to dM and hg is the mean curvature of dM. o is a semi-linear 
Neumann boundary condition. 

More recently, in the study of a near field reflector problem |18] one has the boundary 
condition 


( 1 . 2 ) 


Tu{n) = n*, 
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where 0, Q* are two bounded domains in M”, and is the reflection mapping given by 

2Du 


Tu{x) = 


X £ Q. 


\Du\'^ — (u — Du ■ xY ’ 

The boundary condition (11.21) is fully nonlinear, and in [18] it was shown that (II.2p is oblique 
for any admissible solution u. 

The equations associated with (|l.ll) and (II.2j) are Hessian and Monge-Ampere types, respec¬ 
tively. 

The main goal of the present paper is to define a degree theory, along the line of [T3l[Tl| for 
fully nonlinear elliptic operators with fully nonlinear oblique boundary conditions. See Section 
[2] for the statement and Sections [3ll6] for its proof. As applications, in Section [3 we outline 
how our degree theory can be used to prove the existence of solutions of the boundary Yamabe 
problem and the near-held reflector problem. In the Appendix, we collect some properties of 
the Laplace operator A : for s £ [0, 2] on a compact Riemannian manifold, which 

are needed in the body of the paper. 


2. Statement of the main result 

In this section we introduce a degree theory for second order fully nonlinear elliptic operators 
with nonlinear oblique boundary conditions of general form, 

(2.1) F[u] = Du, D‘^u), in H, 

(2.2) G[u] = g{-,u, Du), on clH, 

where H is a bounded smooth domain in Euclidean n-space, M”’, and / £ x M x M” x S^) 

and g £ x M x M”) are real valued functions, 0 < a < 1. Here S"' denotes the n(n-|- l)/2 

dimensional linear space of n x n real symmetric matrices, and Du = {Diu) and D‘^u = [Diju] 
denote the gradient vector and Hessian matrix of the real valued function u. 

Letting [x, z,p, r) denote points in H x M x R” x S^, we shall adopt the following deflnitions 
of ellipticity and obliqueness for operators F and G El- An operator F : C^’“(H) —>■ (^^’“(H) is 
uniformly elliptic on some bounded open subset O of C"^’“(H) if there exists a constant A > 0 
such that for all u G O, X G H and ^ G R"' there holds 

df 


(2.3) 


dvi 


-{x,u,Du,D^u)iiij > A|^|" 


An operator G : C^’“(H) G^'°‘{d^) is uniformly oblique on O if there exists a constant x > 0 

such that for all u G O and x £ dit 


(2.4) 


^{x,u,Du) • 7 (x) > X, 


where 7 (x) denotes the outer unit normal of dQ at x. 
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Let O C be a bounded open set with dO n (F, G)“^(0) = 0. Suppose that F is 

uniformly elliptic on O and G is uniformly oblique on O. We will define an integer-valued 
degree for {F, G) on O at 0 along the line of [T3l fH] . 

Theorem 1. There exists a unique integer-valued degree 

deg: {((F, G), 0,0) : (F,G) and O are as a6oue| Z 
which satisfies the following three properties: 

(pi) deg{{F,G),O,0) = deg{{F,G),Oi,0) + deg{{F,G), 02,0) whenever Oi and O 2 are open 
disjoint subsets of O satisfying (F, G)“^(0) n (O \ {Oi U O 2 )) = $■ 

(p2) Homotopy invariance property; If t ^ {ft,9t) is continuous from [0,1] to G^’“(n x M x 
M”' X 5") X X M X M"), Ft is elliptic on O, Gt is oblique on O (both uniformly 

in t ^ [0,1] j; and dO D (Ft,Gt)“^(0) = 0 for all t G [0,1], then deg((Ft, Gj), G, 0) is 
independent oft. 

(p3) Compatibility with Leray-Schauder degree; If (F, G) is an invertible linear operator and 
O is a neighborhood ofO, then 

deg((F,G),O,0) = (_l)dimE-(F,G) 

where 

(2.5) F-(F,G) = 0 {ix G : -(F[u],GM) = (AiU,0)}. 

Ai<0 


As usual, the basic properties (pl)-(p3) imply immediate consequences which we list below. 

Corollary 2.1. (a) //deg((F, G), 0, 0) 7^ 0, then there exists u G O such that (F[u],G[u]) = 0. 

(b) IfUcO andUn{F,G)-^{0) = 0, then deg((F, G), G, 0) = deg((F, G), G \ ^, 0). 

(c) If (Fi, Gi)lao = {F 2 , G 2 )\ao, then deg((Fi, Gi), G, 0) = deg((F 2 , G 2 ), G, 0). 

(d) //(F, G)(uo) = 0, (F, G) is Frechet differentiable at uq and (F',G')(uo) is invertible. Then 
deg((F, G), G, 0) = deg((F', G'), F, 0), where O is a neighborhood of uq in G^’"(n) which 
does not contain any other points o/(F, G)“^(0) and B is any bounded open set containing 
the origin. 

The next property is a mild extension of property (p3). 

Corollary 2.2. (e) Assume {F,G) has linear leading terms, (F, G) = (Fi, Gi)-|-(F 2 , G 2 ) such 
that 


(Fi,Gi)[u] = {aij{x)uij-\-bi{x)ui I-c{x)u, {/3i{x)ui + £{x)u)\ 9 n) 
(F 2 ,G 2 )[u] = {fffx,u,Du), b^{x,u)\an), 
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where aij,bi,c € I3i,£ € /* € x M x M"’), and 6 * G C'^’"((9i7 x M). 

Assume that F is elliptic, i.e. (aij) > 0 in id, and G is oblique, i.e. /3 • 7 > 0 on dil, where 
7 denotes the unit outer normal of dil. 

If{Fi, Gi) is invertible, then for any open bounded O G such that dOr\{F, G)“^(0) 

%, there holds 

deg((F, G),0, 0) = (_i)dimE-(Fi,GO degi. 5 .(/d + (Fi, Gi)-1 o {F^, G 2 ), 0, 0), 
where E~{Fi,Gi) is defined as in (|2.5|) . 


3. Definition of the degree 


Consider 

S : G^’^m ^ C“(D) X 

(3.1) u^{Au, {-fiDiU + u)\Qn), 
where 7 is the outer unit normal of dil, and 

T : C^’^{dQ) G^’^idn) 

(3.2) u i-A Atu — u, 

where At denotes the tangential Laplacian over dil. It is well-known that S,T are isomor¬ 
phisms. 

Let F, G be the composite maps as follows, 

(3.3) F = (^F(i),F( 2 )) =SoF : C^’"(D) ^ C“(D) x G^’^idQ) 

(3.4) G = ToG : G^’^m ^ G^’^idn). 


Since S and T are isomorphisms, (F, G) = 0 is equivalent to (F, G) = 0. We are going to 
define a degree for (F, G) by defining a degree for (F, G). 

As in m , we write 

(3.5) F(i)[u] = ast{x,u, Du, D‘^u)Diistu + G^{x,u, Du, D'^u, D^u), 

(3.6) F( 2 )[u] = {ast{x,u,Du,D‘^u)Dsuu-fi +E^{x,u,Du,D‘^u))\g^, 

(3.7) G[u]= {bi{x,u,Du)ATiDiu) + H^{x,u,Du,D‘^u))\Q^, 


where 


Qf 

ast{x,u,Du,D‘^u) = —— (x,u, Du, D'^u), 
dr St 

dg 

bi{x,u,Du) = --— (x,u,Du). 

Opi 


To freeze coefficients in (I3.5p - (l3.7p . we make use of the following result, whose proof is 
postpone until Section [6l 
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Theorem 2. Let agt S G where 1 < i,s,t < n. Assume that (agt) is 

symmetric and there exists A > 0 such that ast{x)^i^j > A|^p for all ^ G M”' and x & Q, and 
there exists x > 0 such that bi{x)'yi{x) > x for all x G dLl. For a constant N, define 

■W ^ ^(3)'^) > 

where 

— o>sfDiigfW IV a^fDgfW ^ 

^{2)'^ ~ {0‘stDstiW~fi)\dQ ’ 

-^( 3 )^ = {bi^riDiw) - NbiDiW - Nw)\qq . 

Then there exists some constant Nq, depending only on ||ost, || 6 j||ci,c«, n, A, x such that 
is an isomorphism for all N > Nq. Furthermore, depends continuously on ast,bi with 
respect to the corresponding topologies. 

We are in a position to define an integer-valued degree for {F, G) : O ^ C‘^'°'{TL) x 
N ote that F, G in (9)~(11) can be represented as 

(3.8) GM) = L^^^[u] + 

where ) and = {R^^^, R^^^, R^^^) are 

w = ast{x, u, Du, D'^u)Diistw - Nagfix, u, Du, D‘^u)Dstw, 

^ ^ {^st{x, u, Du, D‘^u)DstiWXi) , 

L^^^w = {bi{x, u, Du)A\T{Diw) — Nbfix, u, Du)DiW — Nw)\qq_ , 

Rf^^w = Nast{x, u, Du, D^u)Dstw + G^{x, u, Du, D^u, D^u), 

^12)'^ = Efix,u,Du,D'^u)\aQ,, 

R^^^w = Nhfix, u, Du)DiW + Nw + Hfix, u, Du, D'^u)\qq_. 

One can see that maps into x (7^’“(90) x According to 

Theorem [2l there exists some positive number Nq such that is an isomorphism for any 

N > Nq. By [H Theorem 7.3], maps C'^’“(n) x C‘^'°‘{dQ) x C'^’“(5n) into C®’"(n), 

and its norm as a linear map between these spaces is bounded by a constant depends only on 
lla^f ||(; 72 ,a, || 6 j||(; 3 .c«, A and x- It follows that 

is a compact operator from O to C'^’"(f7). 

Moreover, {F,G)[u\ = 0 is the same as n + = 0, i.e., 

do n {Id + (0) = 90 n {F, G)-^ (0) = 0. 
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Therefore, we can define the degree of {F, G) as the Leray-Schauder degree of the map u i—>■ 
u + [u]. (See e.g. |20] for the dehnition of the Leray-Schauder degree.) More 

precisely we have the following dehnition. 

Definition 1. Let {F,G) and O be as in Theorem {Jl We define a degree of {F,G) on O at 0 
by 

(3.9) deg ((F, G), 0,0) = deg^.s^Id + 0, 0), 

where N > Nq and Nq is the constant in TheoremlM 

Note that, by the homotopy invariance of the Leray-Schauder degree, the degree above is 
independent of > Nq. In Sections H] and [5] below, we shall prove Theorem [2] and verify the 
above dehned degree (j3.9p satishes the required properties (pl)-(p3) in Theorem [TJ 


4. Some boundary estimates 


We start with some boundary estimates for linear elliptic systems with oblique boundary 
conditions. We use Bfi to denote {x G M” : |x| < r,Xn > 0} and let T = {xn = 0} PI Bi. Fix 
some integers m > 1 and n > 2. In the sequel, repeated Roman indices are summed from 1 to 
n and repeated Greek indices are summed from 1 to m. Consider the system 

(4.1) a’ffDstu^+ df = y in R+, 

(4.2) bfDiU^ = on T. 

We assume that aff is uniformly strongly elliptic in B^ i.e. there exists A > 0 such that 

C it > for all x G R+ and ^ G 
and 6 “^ is uniformly oblique along T, i.e. there exists x > 0 such that 

—bff{x)rf r]^ > x|i?|^ for all x G T and i] G M™. 


Lemma 1. For m > 1, assume that aff G W^'°°{By is uniformly elliptic and symmetric, 
ds^ G L^{By, c“^ G L^{By and 6“^ G IF^’°°(r) is uniformly oblique along T. Let F = 
(/“)((Li € L‘^{By, G = (5“)™=! e i^(r), and U = (u“)((Li G ^^(^i^) he a solution of the 
oblique boundary value problem (HI])-(1121). Then we have the estimate 


(4.3) II-^^IIl 2 (s+ ) + ll-^^llL2(rnBi/2) ^ ^ ll■^llL 2 (s+) + l|G'llL 2 (r) + \W\\l'^(b+) 


where C depends only on nuj.^ ii^i.ooj-^+p 
ellipticity and obliqueness constants A and x- 


afi I 




„a/3| 


Q:/3 I 




and the 


Proof. We adapt the proof of the well-known Rellich identity for harmonic functions. 

Fix 0 < ri < r 2 < 1. In the sequel, G will denote some positive constant that may very from 
line to line and depends only on the coefficients of the equation. 
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Select a smooth cutoff function (p satisfying 


(/? = 1 in Bj.^ (0), 

y? = 0 on - S(^^+^ 2 )/ 2 ( 0 ), 

0 < < 1 , < CA^, 


where 


A = 


> 1 . 


r 2 — ri 


1. Testing (j4.ip against p^U, we have 


(4.4) [ \DU\‘^ < C [ \F\‘^ + CA^[ \U\‘^ + Ck[ + R-^ [ 

iB+ Jb+^ Jb+^ 4rnBr2 JrnBr. 

for large iii > 0. 

2. Testing (14.ip against p'^DnU, we have 

- [ p^r DnU^ < [ p^DnU^Dsuf^pP + [ p^PjDD^tU^ 

JBf JT JBf 

j p\DU\^ + C [ p^\U\^ 

Jb+ JBt 


|DC/p 


+ CA 


1 1 

< [ p^DnU'^DsU^pi + f p'^Dnilrpt DsU^DtuP 
Jr Jb+ 2 

[ p\DUf + C [ p^\u\^ 

Jb^ Jb^ 


+ CA 


< [ p^Dnu'^DsU^pi - ^ [ p^\DUf + CA [ p\DU\^ + C [ p^ \UP 
Jr ^ Jr JBt JBt 


thus, 

(4.5) 


[ IDU]'^ <Cmax\o, [ p'^Dnu'^DsU^pi}+C [ \F\‘^ + C A [ \DU\‘^ + c[ \Uf 

JrnBr-i ^ Jr ’ JBt^ J Bt^ JBt^ 

3. Let 6n°^ be the inverse of bn^, i.e. bn'^ bP = 5^^. Set u" = bP YPZi DiU^. Then 

Dnu^ = bfg^-v^. 

Testing (14.ip against p‘^V = {p'^v°‘p^i, we have 

-[ p^rv^ = -l {pP^Pj)Dstuf^ - I pP^dfDkuf^ 

JBt JBt JBt 

= [ p^DsU^v°‘p^ + f p^p^Dtv°'DsU^ 

Jr JBt 

(a [ \DU\‘^dx+ [ \U\‘^dx) 

^ Jb+ Jb+ 2 


+ 0 
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Noting that 


This implies that 


n—1 

<ht Dt{bV bj^ DiU>^) 


i=l 


ip‘^DsU°'v°‘asn 


<CA-^ / \F\‘^ + CA 

B+^ 


'B 


\DUf 


(ri+r 2)/2 


Hence, by the nniform obliqueness we obtain in case the integral on the left hand side below 
positive that 

j ip^DsU°‘DnU°‘asn<C j ip‘^DsU°‘{-bn)DnU°‘asn = -C j DsU"^ {g°‘ - v‘^) Usn 

: [ if^\DU\^ + Ce [ |G|2 

Jr JrnBr^ 


< e 


+ CA-^ [ \F\‘^ + CA [ 

Jb+ Jb 


R + 

(ri+r 2)/2 


\DU{^ 


for any e > 0 small. Recalling (14.51) we obtain 

(4.6) [ \DU\^<CA-^[ \F\^ + c[ |Gp 

JrnBr, JbX JrnBr^ 


+ CA 


IB 


\DU(^ 


(j'l+r 2)/2 


Recalling (14.4p with K = e ^ A for some small e > 0, we obtain 


(4.7) 


4. Let 


[ \DUf <CA-^ [ 

J rni?,.. J Bi 


iTr + c 


/rns, 


|Gp 


+ C A' 


f \Uf 

Jb+ 


^2 


+ C€-^A^ 


irnBr 


\U\‘^ + e [ \DUf 
JrnBro 


$(r) = / \DU\'^. 

JrnBr 


Then by ()4.7p with £ = ^, 

$(ri) <^^(r2) + C [ \F\^ + C [ \G\^ + CA^ [ \U\^ + CA^ [ \Uf 
J Jb+ Jr Jb+ Jr 


(4.8) 


■*1 

1 c 


(r 2 - ri)3 


[ |F|2 + C / |G|2+ / \u\^+[\uf 

Jb+ Jr Jb+ Jr 




A standard iteration (see e.g. [5l Lemma 1.1]) leads to 


cl>(-) < CX 


In other words 








NONLINEAR OBLIQUE OPERATORS 


9 


[ \DU\‘^<c[ |F|2+ C / |Gp+ C / |C/p + C 

irnRi/2 Jb^ Jr 


Combining with ()4.4p we conclude the proof. 


□ 


As a consequence, we obtain the following boundary estimate for scalar oblique boundary 
value problems. 

Lemma 2. Assume that Ust £ is uniformly elliptic and symmetric and bi € 

is uniformly oblique along dQ. Let g € Lp‘{dLl) and let w G be a solution of the oblique 

boundary value problem 

(4.9) OstDstw = 0 m n, 

(4.10) biDiW + w = g on dO.. 

Then we have the estimates 

(4-11) ll^llL 2 (afi) + \\Dw\\l2(^qq'j < C\\g\\i2(^QQ'^, 

(4-12) \\D‘^M\L^ian) < C\\g\\H-i-{an)j 

where the constant C depends only on ||as 4 ||^i,oo(Qp and the ellipticity and oblique¬ 

ness constants A and x- 

Remark 1. Later on, we will use the following consequence of (I4.12p : 

(4-13) \\'^TWi\\L^(dn) < C (\\y^^bj\/TWj\\L^(dn) + IlffllL^oo) 

3 

for all i = 1,... ,n, where Vt denotes the covariant derivative along dLl. 


Proof. The proof is based on the local boundary estimate in Lemma [T] 

For every point xq G dLl, we can hnd some sufficiently small r > 0 and a diffeomorphism 
4> : 12 n Br{xo) B+(0). 

Define 

(4.14) w{y) ■.= w{^~^{y)) , (y G S+). 

It is straightforward to check that w satisfies 

f dsiDstw + ds DsW = 0 in 
\ bi DiW w = g on r = {yn = 0} n Bi. 

where a = a is uniformly elliptic, b = (Zl $)^6 is uniformly oblique, and d = Ostdst^. 


Proof of (14.lip : Apply Lemma [D to (j4.15p by setting m = 1, F = 0 and G = g — w, we have 



\Dwf + 



\Dwf < C 
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Consequently 

(4.16) f lUinp + f lUinp < C ( f \g\‘^ + f \w\‘^ + f \w 

JdnnB^/2 JnnB^/2 KJdnnB-r JdnnBr JunBr 

Since dil is compact, we can cover dQ with finitely many small balls Bj.{xi), Xi € dVt, 
z = 1, • • • ,N. Summing the estimates for i from 1 to N, we obtain that 

[ \Dw\‘^+[ \Dw\^<c([ \g\^+[ \w\^+[ |u;|2V 
Jan Jur \Jdn Jan Jnr / 

for a small constant r > 0, where 11^ = {x € 11 : dist(x,cin) < r}. Together with interior 
gradient estimates for (I4.9p . we arrive at 

(4.17) [ \Dw\'^ + [ \Dw\^ < C ( [ |5p+ / \wf+[\w\'^ 

Jan Jn \Jan Jan Jn 

We can then apply a standard argument using compactness and the uniqueness of the problem 
(I4.9l) - (l4.10p to simplify estimate (I4.17P to 

(4.18) / \Dwf + [ \Dw\^ < C [ \g\^. 

Jan Jn Jan 

This finishes the proof of (14.lip . 




Proof of (I4.12p : As before, we hrst investigate (14.151) . Fix some r = l,...,n — 1. By differenti¬ 
ating equation ()4.15p with respect to Xt, one has 

(4.19) (XgiD T d]^D]^{^D'j-w^ — D'j-{Qjgt)Dgiw {^D'j-d]^')D}^w. 


Also, from (I4.15p . we can write DnnW as a combination of {Dgtw : (s,t) / {n^n)} and {D^w : 
1 < fc < n}. Thus, W = {Dt-w)^z\ satisfies 

(4.20) dstDstW^ + dY'DkW^' = r 


for some smooth d]Z' and F = (/’^)”=j{ satisfying |F| < C\Dw\. 

Applying Lemma [T] to (I4.20p , we have 

\\DW\\L2(^rnBy2) < C AhF||L2(r) + 11^^11^2(5+) + ||bF||L2(r) 

Returning to w and using the compactness of dZl and estimate (I4.18jl . we obtain that 

[ \D\/tw\^<C\[ \Y^bjVTWj\‘^+ [ b|2|. 

Jan \Jdn ■ Jan I 

By the equation, is also under control. (14.121) is proved. 


□ 
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5. Proof of Theorem [T] 

We assume for now the correctness of Theorem [2l whose proof will be carried out in the next 
section. Then the degree deg((T, G), 0,0) in Definition [T] is well-defined. Properties {pl)-{p2) 
follow from the properties of the Leray-Schauder degree. For (p3), we prove the more general 
statement in Corollary 12.21 To this end we use the following lemma on the semi-finiteness of a 
linear operator. 


Lemma 3. Assume Uij G 6*, c G /3j G and I G L°°(9D). Assume 

furthermore that {aij) > XI inCt and fd-'y > x on dVt for some positive constants A and x- Then 
there exists p* depending on ||aij|| vi/f°°(o)? II^*IIl°°(o )7 l|c^llL°°(n); II/3i||iyi.°°(an)7 ll^~II l°°(90); 
and X such that for any p > the problem 


(5.1) 


J Oij (x) Uij + hi{x) Ui + c{x) u = pu in Q, 
( fdi{x) Ui -|- i{x) u = 0 on dH 


has no non-trivial solution in H'^{TL). 


Remark 2. If > 0 on dll and the coefficients are smooth, the result follows directly from the 
maximum principle. In fact, /i* can then be taken to be ||c"*'||c' 0 (q). 


Proof. We use energy method. Assume that u G H^{Il) is a solution to dSU). We will use 
C to denote some positive constant which may vary from lines to lines and depends only on 
||ajj||i 4 /i,oo(Q), ||6i||L°°(Q)) lie"'" 11 LOO (Q), ||A||wi’°°(ao), IK~||L°°(afi)) ^ and x- In particular, C is 
always independent of p. 

Multiplying the first equation in (15.ip by u then integrating over D, we get 

p dx < —C~^ / \Du'^ dx + C / dx + / uuij Ui Xj da{x). 

Jo, Ju Jo, J do. 

To proceed, we write 

aijXj = pPi + Xi 

where p = and X-y = 0. Note that p > 0 is bounded thanks to ellipticity and obliqueness. 

It follows that 


C ^ f dx + {p — C) f dx < f u 

■Jfl J S7 J dfl 


p/3iUi + XiUi 


da{x) 


(5.2) 


/ u[—pIu + X{u)]da{x) 

Jon 


Note that, by Stoke’s theorem, we have 
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Returning to (15.2p we hence get 

C~^ f \Du\‘^ dx + — C) f u^dx<C f da{x) 

Jo, Jq j do, 

<e / \Du'^dx + Ce / V? dx 
Ju Jq 

for any small e > 0. Here we have used the compactness of the embedding H^{Q) ^ L^{dQ). 
The assertion follows by choosing e = D 


Proof of Corollarv \2.S[ As before, set {F,G) = {S o F,T oG) : x C^’°‘{d^l) x 

where S and T are given by ()3.ip and ()3.2I) . 

In case (F, G) has the above special form, the operator L = Lu,n ■ 

X G^’°‘{dQ) dehned in (13.81) takes the form 


Lw = ( OstDiistw - NastDstw, {ostDistw , {hi /^T{Diw) - N hi DiW - Nw) 
V 9f2 


dQ 


By Theorem [2l we can select N sufficiently large such that L is invertible, L ^ o (F, G) : 
—)■ is of the form Id + Compact and 


deg((F, G), 0, 0) = degL.sXL-^ o (F, G), 0, 0). 


Set (Fi, Gi) = (FoFi, roGi) : G^’"(H) —>■ G“(H) x G^’"(cIH) x G^’°‘{dQ). By our hypotheses, 
(Fi,Gi) is invertible. Thus, by the product rule of the Leray-Schauder degree, 

deg((F,G),O,0) = J]degi,5.(^"'o(A,Gi),W,0) deg^. 5 ,((A, Gi)-'o (F,G), 0,W), 

u 

where the summation is made over the connected components of G^’“(H)\(Fi, Gi)“^o(F, G){dO\ 
It is evident that deg^, ^ (F“^ o (Fi,Gi),G,0) = 0 if 0 ^ W. Hence 

deg((F, G), 0,0) = degi. 5 ,^i), 0,0) degi. 5 .((i"i, Gi)"' o (F, G), 0,0), 

where 0 is the connected component of G^’“(H) \ (Fi,Gi)“^ o {F,G){dO) containing 0. As 
(Fi, Gi)“^ o (F, G) = (Fi, Gi)“^ o (F, G) = Id+ (Fi, Gi)“^ o (F 2 , G 2 ), it remains to show that 

(5.3) d:=degi,5.(i"'o(A,Gi),0,O) = (_l)dimE-(Fi,Gi)^ 


Define (F 3 ,G 3 ) : G^’“(H) ^ G2’“(H) x G^’^{dn) by 

(F 3 [w], G 3 [tc]) = ( aij Wij ,{PiWi + w] 


dn 


For 0 < t < 1, define Lt : G^’“(H) ^ G"(H) x G^’“(aH) x G^’^paH) by 


dF^lw] 


Ltw = (^(1 - t)astWiist + tAF3[w] - IVF3[rt;], (^(1 - t)astWsu7i + t 
(^(I - t)biAT{Diw) + tArGslw] - A^G3[rt;]^ 


dn 
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We can then apply the proof of Theorem [2] to see that, for sufficiently large N, Lt is an 
isomorphism for each t G [0,1]. Furthermore, as Li — (Ti, Gi) : —>■ x x 

C^’“(9n), o G is a legitimate homotopy for the Leray-Schauder degree. It follows that 

(5.4) d = degi. 5 ,(^"' o (i"i, Gi), 0,0) = deg^, 5 .(ir' ° (A, Gi), 0,0). 


Next, set 

Ltw= (AF3[u;]-(l-t)A^F3[u;], 


dF[w] 

d'f 


+ tF[w]J (^AtG 3 [u;] - ((l-t)A + t)G 3 [u;]^ 


dQ 


Arguing as before, we have Lt is invertible and 

(5.5) 

degL.sXLi^ ° (i"i,Gi),G,0) = degL.siLo^ o (Fi,Gi), G, 0) = deg^.^X^r' ° Gi), G, 0). 

Note that Li = (S' o F^,T o G 3 ) and so o {Fi,Gi) = {F^,Gs)~^ o (Fi,Gi). Hence, by 
(15.41) and (|5.5p 

d = degi.5.((i"3,G3)-'o(Fi,Gi),G,0) =deg^.5.((i"i,Gi)-io(F3,G3),G,0). 


Set 

At = (Fi, Gi)-i o [(1 - t)(F 3 , G 3 ) - t{Id, 0 )], 

where {ld,0) is considered as an operator from G'^’“(fi) into G^’"(i4) x G^’“((9fl). By the 
maximum principle and obliqueness. At is a continuous family of invertible linear operators 
acting on G^’"(fi). Moreover, for t G [0,1), (1 — t)“^ At is of the form Id + Compact. Hence, 
by the homotopy invariance property of the Leray-Schauder degree, 

d = deg((l — t)~^At, G, 0) for any t G [0,1), 

which implies 

d = (_l)dimE-(A0 ^ ^ 

where 

E-{At)= 0 {uGG^’“(H) :Atu = At(t)u}. 

Xi{t)<0 

To proceed, we claim that there exists some G > 0 and 6 G (0,1) such that, for any t G 
(1-5,1] 

(5.6) — G < A < — — for any negative eigenvalue A of At. 

0 

Indeed, let A be an eigenvalue of some At and u be a corresponding eigenfunction. Since At is 
invertible, 

J Uij Uij + hiUi + cu = — t)aij Uij — tu] in Ll, 

\ /3iUi+ju = j{l- t){Pi Ui + u) on dLl, 
which is equivalent to 

f Uij + bi Ui + cu + jz^u = 0 in H, 

\ PiUi + j^^lu-jX^u= ondQ, 
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It is readily seen that the first inequality in ()5.6I) follows from the invertability of {Fi, Gi) while 
the second follows from Lemma [3] for 5 sufficiently small. 

By (15.61) and the compactness of Ai , we can pick a (simply connected) neighborhood M of 
[—C, — ^] in the complex plane such that in the set of eigenvalues of Ai lying in J\f consists of 
all negative real eigenvalues of Ai. Furthermore, we can assume that J\f is symmetric about 
the real axis. Set 

E{At,Af)= 0 = Ai(t)w}, 

E*{At,M) = 0 e ■.Atu = Ai(t)u} 

Xiit)eJ\r\R 

By the continuity of a finite system of eigenvalues (see e.g. m pp. 213-214]), for 5 > 0 
sufficiently small, 

dimE{At,N') is independent of t £ (1 — <5,1]. 

Also, since At has real coefficients. 


dimE*{At, J\f) is even. 


Therefore, by (|5.6I) . 

d = lim( —= lim( — = ('_l')dimE(Ai,A/') _ (•_]^)dimE-(Ai)_ 
As Ai = —{Fi,Gi)~^ o (/d, 0), (15.31) follows. The proof of Corollary 12.21 is complete. 


□ 


Finally, we prove the uniqueness of the degree under properties {pl)-{p3). We will only sketch 
the argument since it is standard. Let d{{E,G),0,d) be a degree which satisfies {pl)-{p3). 
We will show that d{{F,G),O,0) = deg{{F,G),O,0). First, by Smale’s infinite dimensional 
version of Sard’s theorem, there exists /o G (^^’“(fi) and go £ G^’‘^{dF,) such that all zeroes of 
{F - fo,G - go) are non-degenerate and ||F[ri] - t/o||c2,«(n) + II^M - ifi'o||c3.“(9Q) > cq > 0 
for all u £ dO, all t £ (0,1) and some cq > 0. By the homotopy invariance property (p2), 
d{{F, G), 0, 0) = d{{F -fo,G- go), 0, 0) and deg((F, G), 0,0) = deg((F - fo,G - go), 0, 0). 
Thus, we may assume from the beginning that all the zeroes of {F, G) in O are non-degenerate. 
The uniqueness then follows from the addition property (pi). Corollary 12.1 I d), and the degree 
counting formula (p3) for linear operators. 

We have finished the proof of Theorem [1] □ 


6. Proof of Theorem [2] 

We start with the injectivity of L^: 

Proposition 6.1. Under the hypothesis of Theoreml^ there exists some constant No, depending 
only on ||as 4 ||ci,a, \\bi\\(^i,a,n,\,x such that for all N > No, is injective. 
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Proof. Step 1. If astWst = 0 in Q and = 0 on (90 simultaneously for some > 0 sufficiently 

large, then V := bi DiW + rc = 0 on dPt, hence w = 0. 

Integrating by parts, we have 


( 6 . 1 ) 


= - [ (biArWi) ('^bjDjW+ 10) + N f |l/p 
Jan ^ Jan 

>f \y^biVTWi\^ + N [ | 1^|2 
Jan • Jan 

— e\\VTDw\\‘j^2(^QQ^ — Ce {\\Dw\\\‘^{dn) + ll'“^lli2(ao)) > 


for any positive constant e, where the constant > 0 depends on e and H^jUci.c. Here Vy 
denotes the covariant gradient operator on dO,. 

By Lemma El we have 


Ian 


Ian 


\Dw\^<c[ |H|2, 

Jan 

\^TWi\^ <c [ (y^b.VTW.V + |Hp. 

Jan ^ ^ 


Thus, by choosing e > 0 sufficiently small in (16.lip , we deduce that 


( 6 . 2 ) 




{biDrWif + y 



{biDiW + w)^ . 


This implies that biDiW + re = 0. Since agtDstw = 0 in fl, we obtain rc = 0 from the maximum 
principle. Step 1 is proved. 


Step 2. For any w satisfying = 0, there holds 

(6-3) WwWH^n) < C iWastWsth^in) + I|w^IIl2(o)) , 

(6-4) ll^i’llR'3(Q) < C (||asiu;st||Hi(f2) + > 

where C depends on ||ast||(^i.Q, ||6i||co,c., re, A and y. 

Indeed, = 0 implies that 

(6.5) LtV := -AtV + NV = g on dQ, 

where V = biWi + w as in Step 1 and g = {ATbi)wi + 2Vr6i • ^TWi + Atw. The inverse 
operator LfJ : H^{dPt) is a bounded operator and ||Ty^|| < C for some constant 

C independent of N (see Lemma [5] in the appendix). When s = 1/2, by the trace theorem and 
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interpolation we have 

11^11^1/2(9^) ^ C'||s'||j^-3/2(9Q) 

< C[\\w\\H^/^{dn) + ll-^'“^llLI-i/2(9n)] 

^ ^[||//^lliLi/2(9C) + ll-^'“^lliLi/4(9n)] 

< C'||u;||jy7/4(Q) < £\\w\\h2(^q) + C'e||ii;||i 2 (f 2 ), 

for any small e > 0. Similarly, when s = 3/2 we have 


( 6 . 6 ) 


(6.7) 


11^11/13/2(911) < £||/1^II//3(D) + C'£||/1i||l2(D)- 


From estimates for linear elliptic equation of second order with oblique derivative 

boundary condition (see e.g. [U Theorem 15.2]), we have 

l|/i^ll//2(n) < C (^||as4'«;s4||j;,2(Q) + ||F||j^i/2(9q) + ||'»i||l2(q)^ , 

l|//^llf/3(Q) < C + 111^11^3/2(911) + l|//'llL2(r2)^ . 

Recalling (|6.6h and (j6.7h we arrive at (16.31) and (16.4j) . 


Step 3. For any w satisfying = 0, there holds 


( 6 . 8 ) 

(6.9) 


|//^lli/2(o) < C\\astWst\\L^{n)^ 
|//^lli/3(o) < CII astir’s! II i/i(D)- 


We will only derive (16.8p . The proof of (|6.9p is similar. 
By (16.31) . it suffices to show that 


I|w^IIl2(d) < C\\astWst\\L^(n) for all w e Ker{L^^~^) 
Otherwise, there is a sequence G Ker{L^^^) such that 


/«^”^IIl 2 (o) = 1 but \\astwf^’\\L 2 (^^) < - 


ip) I 


n 


By (16.31) . ||irfo) ||j|t 2 (Q) < C, thus converges weakly in and strongly in to some w*. It 
is straightforward to show that Ost ir*^ = 0 in fl and = 0 on 512. By Step 1, w* = 0. This 


contradicts ||irfo^ ||^ 2 (ft) = 1 and the strong convergence of to w* in L^. This concludes 
Step 3. 


Step 4. L^w = 0 implies ir = 0. 
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Using L^i'^w = 0, we have 

0 = ^ i-akiWki) dx 

(6.10) > Cl / \D{astWst)fdx + N \astWst\'^ dx - C2 \\w\\h3(^q)\\w\\h2^q) 

J Q, J Q, 

>^ / \D{astWst)\‘^ dx + {N - cs) / \astWst\'^ dx, 

^ Jn Jn 

where the constants ci,C 2 ,C 3 depend only on ||ast||ci,c., || 6 i||(; 7 i,c«, n, A and y. In the above, the 
hrst inequality follows from integration by parts, = 0, and Holder’s inequality, while the 

second inequality follows from the Cauchy-Schwarz inequality and (I6.8l) - (|6.9p . It follows from 
(I6.10p that for N sufficiently large, we must have astWst = 0. By Step 1, this implies tc = 0, 
which completes the proof. □ 


We now turn to proving the surjectivity of . Consider Lq : ^ xC^’“(5H) x 

defined by 

Lqw = {/\^w - NAw, ( 7 * A Arc)) Igfj, ( 7 * At (Diw) - Nji Diw - Nw)\qq) . 

Note that (1 — t)LQ + coincides with when agt and bi are replaced with (1 — t)6st + tost 
and (1 — t) 7 i -|- tbi, respectively. Therefore (1 — t)LQ + is injective for all t € [0,1]. By 
the continuity method [U Theorem 12.5], is isomorphic if and only if Lq is isomorphic. In 
other words, it suffices to establish the surjectivity of Lq . 

Define ^ C"(D) x x C^’^{dn) dehned by 

L^w = {A'^w - NAw, {'yiDiAw))\Q^ , {ArijiDiW+ w) - N{jiDiW+ w))\q^) , 

and let = (1 “ AtL^ . It is easy to see that is bijective. By the continuity method 
[H Theorem 12.5], in order to show that Lq is bijective, it suffices to show that is injective 
for all 0 < t < 1. This is a consequence of the following lemma: 

Lemma 4 . If Aw = 0 in D and if 

Mtw := (1 — t)AT {'jiDiW -|- re) -|- 17^ (Diw) — A^(7i DiW + w) = ^ on dVl 
for some 0 < t < 1, then V := 7* Diw + tc = 0 on dQ, hence w = 0 . 


To see this, we adapt the argument in Step 1 of the proof of Proposition 16.11 We compute 


0 = — / Mtw V 

Jan 


( 6 . 11 ) 


> 


(1-t) f jVTVl^-tf {'nATDiw)V + N [ jUp 
Jdn JdU JdQ 

[ \VtV\‘^ + N [ JUJ^ 

Jan Jan 


£\\VtDw\\12^q^^ - Ce 


h{an) + 11 ^"^ 
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for any positive constant e, where the constant Cg > 0 depends on e and || 7 i||ci,c.. By Lemma 
[21 we have 

[ \Dw\^<c[ 

Jan Jan 

[ \VtDw\‘^<c[ \VtV\‘^+ \V\‘^. 

J do. J do. 

Thus, by choosing e > 0 sufficiently small in (16.111) . we deduce that 

(6.12) 0>i/ iVT^P + y/ 

^ Jan ^ Jan 

which implies that 1^ = 0 as desired. This concludes the proof of the lemma and hence of 
Theorem [21 □ 


7. Some applications 


7.1. Boundary Yamabe problem. Let {M,g) be a smooth Riemannian manifold of dimen¬ 
sion n > 3 and with non-empty boundary dM. Define the Schouten tensor 

^9 = ^ {^^^9 - 2(n-l)^5 a) , 

where Ric^ and Rg are respectively the Ricci curvature and the scalar curvature of g. Let hg 
denote the mean curvature of dM. In conformal geometry, one is interested in Ending a positive 

4 

function u such that the metric gu '■= u^-^g such that 


(7.1) 


f{X{AgJ) = l,X{AgjGr, 


V = c, 


where c is a given constant in M, T C M” satisfies 


T is an open, convex, symmetric cone with vertex at the origin, 

Ti := {A G ^ A* > 0} D r D r„ := {A € R” : A* > 0}, 
and / G (^““(r) D (^^(r) satishes 

/ > 0 in r and / = 0 on dT, 

fxi > 0 in r. 

We refer readers to m for the literature on this problem. 

In [13], an existence theorem for ()7.ip was established using a degree theory which is less 
general than the one considered in the present paper. We outline some of the arguments here. 

First, we restrict ourselves to the case where 


and 


g is locally conformally flat. 


dM is umbilic. 
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Under the assumption that 


x{Ag) G r, 


it was shown in m that all (positive) solutions of (j7.ip . if exist, satisfy 


lnrx| < C{M,g,f,r,c). 


Under an additinal assumption that 


/ is concave and c > 0, 

one then appeals to known local and estimates, and Evans-Krylov’s theory to conclude 
that all solutions of dEU) satisfy 

\\lnu\\ci,c.(M) <C{M,g,f,r,c,a). 

It should be noted that, when c < 0, estimates fail at both local and global levels. See 

[Hill]. 

With the above a priori estimate, our degree is defined and independent along a homotopy 
connecting (/,E) to ((TijEi). Here ai is the first elementary symmetric function. By property 
(e), the degree of (aijEi) is the same as the Leray-Schauder degree for (iTi,ri), which was 
computed to be non-zero in [7]. The desired existence result is established. 


7.2. Near-field reflector problem. Consider the Monge-Ampere type equation arising in a 
near-held reflector problem |10l [T8] 

(7.2) p*{Tu) detDTu = p{-) in H 


with the boundary condition 

(7.3) Tu{n) = n*, 


where p, p* are, respectively, the intensities of incident and reflected rays satisfying 


(7.4) 




= I P 
n* 


Q, Cl* are two bounded domains in M"" with ^ Hi(0), and Tu is the reflection mapping 

2Du 


(7.5) 


Tuix) = 


\Du\'^ — {u — Du ■ xY ’ 


X G 11. 


In [18] we proved the existence of solution for (17.21) and (j7.3l) under some convexity assump¬ 
tions on domains Cl and Cl* by using the degree theory established in Section 2. We outline the 
main steps here and refer the readers to [18| for more details. 

First we show the a priori estimate that under suitable convexity and smoothness assump¬ 
tions, the boundary condition ()7.3p is strictly oblique, and moreover, for any classical solution 
u of ()7.2p - ()7.3p . we have the a priori estimate < C for a G (0,1) and a positive 

constant C depending on the given data. 
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To construct the homotopy family, we use the domain foliation, namely there exist an in¬ 
creasing family of smooth domains such that flo = Br{0), Oq = Tuq{^o)', rii = fl, 

Q* = 0*; and are uniformly convex and y*-convex, respectively, for all 0 < t < 1. 

Consider the family of problems 

(7.6) Ft[ut] = det [DTu,] - \-^^ + il-t)det[DT^A =0 mQt, 

Gtlut] = (fit oT(-,ut,Dut) = 0 on dfit, 

where e > 0 is small, and <fi* is the dehning function of fl*. Let := {^t : M” —>■ M”} be a 
family of diffeomorphisms such that ^t(flt) = Bi(0) for each t € [0,1] and ^t S uniformly 
with respect to t. Dehne 

(7.7) Ft[u] = Ft[uo in Bi(0), 

Gt [n] = Gt[uo on dBi (0), 

for any u € C"^’"(i?i). 

Let O be a bounded open set in {u G C^’“(i3i) : ||u||^ 4 ,c.cgj) < G{Ge + 1)} such that Ft is 
elliptic and Gt is oblique on O, and dO fl {Ft,Gt)~^iO) = 0 for all t G [0,1]. From the initial 
construction, uq is the unique solution of (Fo,Go)[u] = 0. By the properties of the degree, 
deg((Ft, Gt), 0,0 ) is dehned for 0 < t < 1 and 

deg((Fi,Gi),O,0) =deg((Fo,Go),0,0) 

=deg((Fo,Go),Oo,0 )/O, 

where Oq = {u G G^’‘^(flo) : ||i^||(74,a(no) < Ce + !}• This implies that there exists a solution 
Ue G C^’"(Bi) of the boundary value problem (|7.7I) at t = 1. Hence there exists a solution 
Us G C"^’“(H) of the boundary value problem 

(7.9) det [DTu] = 

^ ^ ^ ^ P*{Tuy 

Tuifl) = fl* 

for arbitrary small e > 0. To complete the existence proof we now need to let e —)• 0. Write 
equation ()7.9ji in the form of 

(7.10) p*{Tu) det DTu = e^(“-““)p(-) in fl. 

Let {us} be the family of solutions of the problems (I7.10p . From (17.4p 

p = [ p*= [ 

Jn* Jn 

we see that u^ — uq must be zero somewhere in fl. Hence, from the a priori estimates 
is bounded independently of e. Thus a subsequence of {ug} converges in C'^’^(H) for 0 < /I < a 
to a solution u solving (IZSD-dZSl), as required. 







NONLINEAR OBLIQUE OPERATORS 


21 


Appendix A. Some well-known properties of the Laplace operator on a 

COMPACT MANIFOLD 


Let {M,g) be a compact Riemannian manifold and A denote the Laplace operator of g. Let 
= H^{M) denote the Sobolev space of some real exponent s. It is well-known that —A 
maps into In this appendix, we collect some well-known properties on the spectrum 

of —A : for s € [0, 2] which are needed in the body of the paper. 

For s = 0, = L^. For s = 2, is defined as the set of space of functions whose first 

and second derivatives also belong to L^. is defined as the dual of H^. For s € (0, 2), 
can be defined as an interpolation space of and as follows (see e.g. my- 


= \u = 


|u = ^ Ui^ L'^ Ui€ ^ (2 “||uj||^2 + 2*^^ *^||wi||^2) < oo|, 0 < s < 2. 


For u G , its Ll^-norm is defined by 


where the infimum is taken over all possible represenation u = '^Ui. For s G (—2,0), LI® is 
defined as the dual of which is the same as the following interpolation space of H~^ and 


L2: 


= \u = 


{-= E 


Ui G H 


-2 


Uj G L 


EP 


-i(2-|s|)| 


U. 


iWH 


- 2+2 


i|s| I 




|2 

Il2 


) < oo}, 


-2 < s < 0. 


Its norm is defined similarly. 

For u G H^, Au G is defined by standard differentiation. For u £ Au G is defined 
by 

(An, v )^-2 H 2 = (u, Av)i ^2 12 for all v G . 

Clearly A : —>■ and A : are bounded linear operator. Furthermore, it is easy 

to check that A maps il® into LL®“^ for s G (0, 2) and is a bounded linear operator between 
these spaces. Furthermore, for ip G C°°, 


{Au,ip)us- 2 ^H 2 -s = {u,Aip)j^ 2 ^i 2 for all ip G C°°{M). 

Next, fix some A > 1 and s G [0,2]. Consider the operator —A + N : il® —)■ LL®“^. We 
claim that this operator is an isomorphism. This is clear for s = 2. For s < 2, note that 
—A + N : LI® —>■ LL®“^ is injective. Indeed, if —Au + Nu = 0 for some u G LI®, then, for any 
ijj G C°°{M), there exists ip G C°° such that (—A + N)ip = ijj hy standard elliptic regularity, 
and so 

= («, (-^ + ^)+)l2,l2 = {{-A + N)u,<p)hs-2^h2-s = 0 
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which implies that u = 0. But as the formal adjoint of —A + N : —)■ is —A + N : 

—>■ H~‘^, which is injective, the Fredholm alternative implies that —A + N : —>■ 

is surjective. The claim is proved. 

Lemma 5 . There exists some constant C such that for any N >1 and s € [0,2], there holds 

||(-A + < C. 

Proof. Consider first s = 2. If (—A + N)v = u G Lp‘, then by integrating by parts and Cauchy- 
Schwarz’s inequality 

WDvWl^ + N\\v\l2 = {u,v)l2^l2 < ^Ikllia + ylblli2, 

which implies 

||L>u||^2 +A^||u|^2 < {u,v)l2^l2 < y||^^|li2. 

This together with standard elliptic theory leads to 

\\D\\\l2 < C(||Au||i2 + ||u||i2) = CdjAu - u\\l2 + ||u||i2) < C\\u\\l2. 

We have thus shown that 

\\{-A + N)-^\\c(L^H^)<C. 

By duality, this gives 

||(-A + A)-dlz:(H-2,L2 )<C. 

Now for a general s G (0,2) and u G we write u = Y^Ui G with Ui G Then 

(—A + N)~^u = X] Uj G with Vi = (—A + N)~^Ui G We then have 

||(-A + < ^(2-||i;d|i2 + 2*(2-)||i;db2) 

Since this is true for all possible representations ix = we thus arrive at 

!!(—A + A) < (7111x11^^8-2, 

which is exactly the assertion. □ 
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